This specification includes a microfiche appendix (consisting of 1 sheet with 51 frames) which contains partial source code listings (Appendices A and C) and an application program interface specification (Appendix B) of a software product that embodies the invention. These materials form part of the disclosure of the specification.
This source code listings and screen displays of this specification are subject to copyright protection. The copyright owner has no objection to the facsimile reproduction of the patent document or portions thereof as it appears in the files or records of the U.S. Patent and Trademark Office, but otherwise reserves all rights whatsoever.
The present invention relates to software tools for load-testing Web sites and other types of client-server systems. More particularly, the invention relates to a method of efficiently generating a load testing scenario that allows a Web site to be tested according to browsing behaviors of typical users.
With the increasing popularity and complexity of Internet and intranet applications, the task of managing Web site content and maintaining Web site effectiveness has become increasingly difficult. Company Webmasters and business managers are routinely faced with a wide array of burdensome tasks, including, for example, the identification and repair of large numbers of broken links (i.e., links to missing URLs), the monitoring and organization of large volumes of diverse, continuously-changing Web site content, and the detection and management of congested links. These problems are particularly troublesome for companies that rely on their respective Web sites to provide mission-critical information and services to customers and business partners.
Several software companies have developed software products which address some of these problems by generating graphical maps of Web site content and providing tools for navigating and managing the content displayed within the maps. Examples of such software tools include WebMapper(trademark) from Netcarta Corporation and WebAnalyzer(trademark) from InContext Corporation. Unfortunately, the graphical site maps generated by these products tend to be difficult to navigate, and fail to convey much of the information needed by Webmasters to effectively manage complex Web sites. As a result, many companies continue to resort to the burdensome task of manually generating large, paper-based maps of their Web sites. In addition, many of these products are only capable of mapping certain types of Web pages, and do not provide the types of analysis tools needed by Webmasters to evaluate the performance and effectiveness of Web sites.
Another problem in the field of Web site and intranet management relates to the ability of the site to handle peak loads. The heavy use of a site can lead to a significant degradation in performance, or even a complete loss of service. Examples of Web sites that have undergone severe performance degradations during heavy use include the IBM Olympic site during the 1996 summer olympics, and the CNN Interactive Election Day site during the 1996 presidential election.
Mercury Interactive Corporation, the assignee of the present application, has addressed this problem by developing software tools that allow companies to load-test their World Wide Web and intranet sites. Mercury Interactive""s LoadRunner(copyright) 4.5 and Astra(trademark) SiteTest 1.0 products, for example, include functionality for sending large volumes of client requests (representing hundreds or thousands of concurrent users) to a site while monitoring the site""s performance. These products use a virtual user executable, or xe2x80x9cVuser,xe2x80x9d to send the client requests to the Web site while monitoring the site""s performance. The Vuser sends the client requests to the site according to a pre-defined test script (also referred to as a xe2x80x9cVuser scriptxe2x80x9d or xe2x80x9cWeb scriptxe2x80x9d), which is in the general form of a list of the HTTP (HyperText Transport Protocol) messages to be sent to the site. In one implementation, up to 50 Vusers can be run concurrently on a single Windows(copyright) or 95 workstation, with different Vusers using different test scripts.
To facilitate the generation of the test scripts, the LoadRunner(copyright) 4.0 and Astra(trademark) SiteTest 1.0 products are provided with a script generation tool referred to as the xe2x80x9cVuser Generator.xe2x80x9d This tool operates by capturing actual HTTP and/or HTTPS (Secure HTTP) traffic between a standard Web browser and the Web server, and recording this traffic into a script file. Thus, for example, to provide a Vuser that repeatedly accesses a particular sequence of Web pages, the user can launch the Vuser Generator, and then sequentially access the Web pages with a standard browser.
To generate a test that emulates multiple concurrent users, the user can generate and save multiple test scripts (e.g., a set of 5 scripts), and then use a xe2x80x9cScenario Wizardxe2x80x9d tool to define how these scripts will be used to test the site. For example, the user can define a group of 10 Vusers that will play back the same script, and can define other Vusers that will play back uniquely-assigned scripts. This test definition (referred to generally as a xe2x80x9ctest scenarioxe2x80x9d or xe2x80x9cscenarioxe2x80x9d) is then stored as a xe2x80x9cscenario filexe2x80x9d that can thereafter be loaded and run to test the site.
While the Vuser Generator tool provides a simple method for generating test scripts, the tool requires the user to actively browse the Web site. This can be burdensome, especially if the Web site is large. For example, if the Web site has 200 pages, the user would normally have to browse all 200 pages to generate a test script (or a set of test scripts) that covers the entire site. The process of generating a scenario can also be cumbersome, especially if a large number of scripts are involved.
In addition, the test scripts and scenarios generated by this method are not necessarily representative of the paths and browsing behaviors followed by typical visitors. (To distinguish between users of the disclosed tools and regular users of the site, the term xe2x80x9cvisitorxe2x80x9d is used herein to refer to the latter.) For example, the resulting load test may heavily stress an area of the site that is rarely accessed, while failing to adequately stress more popular areas of the site.
It is thus a goal of the invention to provide a test generation tool that eliminates the need for the user to browse the Web site or actively define the scenario. It is a further goal of the invention to provide a test generation method that allows the site to be stressed according to typical usage patterns.
In accordance with these goals, a software module and associate method are provided for automatically generating test scenarios (including test scripts) based on information stored within a server access log file. The server access log file (xe2x80x9clog filexe2x80x9d) is preferably a standard-format log file of the type commonly generated by commercially-available Web servers. These log files contain information about accesses that have been made to the site over some period of time (typically a few weeks), including the IP addresses of the visitors, the URLs (Uniform Resource Locators) that were accessed, and the times and dates of the accesses.
In accordance with the invention, the software module implements a scenario generation process that preserves the load distribution (e.g., the distribution of accesses among URLs) represented within the log file. Thus, for example, if, during the time period to which the log file corresponds, a first area of the site was accessed more heavily than a second area, the resulting test scenario will stress the first area more heavily than the second area (according to the same general proportions as within the log file).
In a preferred embodiment, the scenario generation process involves using the IP addresses and timestamps within the log file to xe2x80x9ctracexe2x80x9d the navigation paths taken by individual users. This produces a routes list which includes information about the number of xe2x80x9chitsxe2x80x9d that occurred on each link. The routes list is then translated into a scenario that comprises a set of test scripts (stored as script files) and a scenario file. The scenario can thereafter be loaded and executed, using either the LoadRunner product or the Astra SiteTest product, to load-test the Web site.